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Adult Learning Focused 
Institution Initiative 



Adults are seeking college degrees in greater numbers. Their “nontraditional” characteristics — 
part-time enrollment, full-time employment, financial independence, and parental responsibilities — 
create needs and priorities that differ from traditional students. They need institutional flexibility in 
curricular and support services, academic and motivational advising supportive of their life and career 
goals, and recognition of experience and work-based learning already obtained. 

CAEL addresses these issues through its Adult Learning Focused Institution (ALFI) Initiative, the 
goal of which is to help colleges and universities improve learning opportunities for working adults 
and the employers who support their education. 

Principles of Effectiveness for 
Serving Adult Learners 

At the center of the ALFI Initiative is CAEL’s Principles of Effectiveness for Serving Adult Learners, 
created to assist colleges and universities in meeting the unique needs of their adult students. 

The Principles provide a framework that helps institutions of higher education adopt policies and 
practices to make educational opportunities more accessible and to remove obstacles from the path 
to degree completion. 

The Principles are based on a benchmarking study of six highly adult-learning-focused colleges 
and universities and input from other higher education stakeholders. 
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EIGHT PRINCIPLES OF EFFECTIVENESS FOR SERVING ADULT LEARNERS 



OUTREACH 


The institution conducts outreach to adult learners 
by overcoming barriers of time, place, and tradition 
in order to create lifelong access to educational 
opportunities. 


LIFE & CAREER PLANNING 


The institution addresses adult learners’ life and career 
goals before or at the onset of enrollment in order to assess 
and align its capacities to help learners reach their goals. 


FINANCING 


The institution promotes choice using an array of pay- 
ment options for adult learners in order to expand equity 
and financial flexibility. 


ASSESSMENT OF LEARNING OUTCOMES 


The institution defines and assesses the knowledge, skills, 
and competencies acquired by adult learners both from the 
curriculum and from life/work experience in order to assign 
credit and confer degrees with rigor. 


TEACHING-LEARNING PROCESS 


The institutions faculty uses multiple methods of 
instruction (including experiential- and problem-based 
methods) for adult learners in order to connect curricular 
concepts to useful knowledge and skills. 


STUDENT SUPPORT SYSTEMS 


The institution assists adult learners using comprehensive 
academic and student support systems in order to enhance 
students’ capacities to become self-directed, lifelong learners. 


TECHNOLOGY 


The institution uses information technology to provide 
relevant and timely information and to enhance the learning 
experience. 


STRATEGIC PARTNERSHIPS 


The institution engages in strategic relationships, 
partnerships, and collaborations with employers and other 
organizations in order to develop and improve educational 
opportunities for adult learners. 



CAEL’s next step in the ALFI Initiative was the creation of a workbook entitled, Best Practices in 
Adult Learning: A Self-Evaluation Workbook for Colleges and Universities. The workbook is based on 
the Principles and describes practices and policies used by high-performing colleges and universities 
that serve adult learners. The workbook also offers an opportunity for campus leaders to informally 
assess their institutions. Yet while the stories and lessons are inspiring, by themselves they fall short 
of answering the questions that colleges and universities ask about the means of continuous 
improvement. Thus, with support of Lumina Foundation for Education over a two-year period, 
CAEL worked with its organizational and institutional partners to create surveys that bring useful 
data to answer the question asked by many colleges and universities: “How well are we serving 
our adult learners?” 
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Adult Learning Focused 
Institution Assessment Toolkit 



CAEL designed die Adult Learning Focused Institution (ALFI) Assessment Toolkit specifically 
for colleges and universities that want to retain undergraduate adult learners by providing programs 
that meet and exceed their expectations. 

The ALFI Assessment Tools are a pair of survey instruments that are based on CAEL’s Principles 
of Effectiveness for Serving Adult Learners. The Tools work together to help uncover what is 
important to adult learners as well as pinpoint an institution’s strengths and areas for improvement. 
They provide a powerful tool for colleges and universities to align their institutional policy with 
the needs of adult learners. 

Institutions assemble a cross-institutional team of faculty and administrators to complete approximately 
fifty items in the Institutional Self-Assessment Survey (ISAS), which offers a unique opportunity for 
dialogue and reflection on the institutions resources and practices devoted to adult learners. The ISAS 
scans eight key areas: institutional background; mission, leadership, and organizational structure; 
recruitment and orientation; tuition and financial aid; teaching and learning; faculty recruitment and 
development; student services; and key constituencies. 

The second part of the Toolkit, the Adult Learner Inventory (ALI), is completed by a sampling of adult 
students. Created by Noel-Levitz and CAEL, the ALI measures how satisfied adult learners are and what 
is important to them. These measurements show how to better meet the needs of adult learners as well 
as what issues should receive top priority. When the ISAS and ALI are used in tandem, they provide 
a revealing view of the congruency of perceptions amongst institutional leaders and adult learners. 

The detailed report received from both surveys includes national benchmarking data to compare 
results with institutions serving adults nationwide. Implementation of the ALFI Assessment Toolkit 
also provides institutions the opportunity to become part of the Coalition of Adult Learning 
Focused Institutions. These institutions are recognized for their ongoing commitment to programs 
promoting success in adult learning. Benefits to ALFI Coalition partners include special 
conferences, use of the ALFI logo, and listing on CAEL’s Web site as a Coalition Participant. 
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Testing the Model 

Decision makers can only count on data taken from instruments whose validity is secure. For the ALFI 
Assessment Tools, CAEL worked with partners at Noel-Levitz (NL) and the National Center for Higher 
Education Management Systems (NCHEMS), subject-matter experts as well as with representatives at 
the pilot institutions whose daily work is dedicated to adult learners. This multi-faceted process assured 
that the ALFI Assessment Tools represent the best practices revealed through CAEL’s studies. 

Further evidence of the validity and impact of the ALFI Assessment Tools comes from an independent 
report from evaluator Mary Williams and Associates. According to this report, the ALFI Assessment 
Tools made an impact on the institutions that used them. Among the pilot institutions, 84 percent 
have acted in some way to build stronger capacity to serve their adult learners. 



A View of Adult 
Student Re-enrollments 



Another step in our evaluation model — adult student responses to the ALFI Assessment Tools — also 
shows the efforts stimulated by the Tools lead to positive outcomes. For this work, CAEL asked for the 
assistance of the institutions that participated in the pilot study so that data on student retention and 
success could be correlated with scores on the ALFI Assessment Tools. The purpose of the study was 
to determine if institutions with strong scores from the ALFI Assessment Tools retained adult students. 

For this study, CAEL provided exact time parameters when requesting the data, so that all 
institutions would report on an identical period. CAEL asked for adult student enrollment data 
from a recent period (July 1, 2002 to September 30, 2003) to ensure availability of data. This also 
coincided with the time that institutions were administering the ALFI Assessment Tools. CALL 
also asked how many of the adult students who enrolled within that period then re-enrolled 
within a one-year period. Nearly all of the pilot institutions provided their data (twenty-one out 
of twenty-five institutions). Using this data, CAEL correlated the retention experience with the 
corresponding scores from the ALFI Assessment Tools. 



Limitations of this approach should be noted. First, no links were made to the actual students 
who participated in developing the the Adult Learner Inventory. As a result, one cannot partition 
students’ reasons that would explain institutional retention results. For example, students who start 
course work may not re-enroll for many reasons wholly unrelated to how well the school performs 
as an Adult Learning Focused Institution. Family emergencies, shift changes at work, personal 
dislike of individual instructors, and a host of similar reasons can occur. 

Also, this study could not determine whether or not re-enrollment results are due to improvements 
that institutions implemented following use of the ALFI Assessment Tools. Accordingly, the 
appropriate way to view the study’s retention results is as an effort to provide initial evidence of 
the concurrent validity (but not predictive validity) of best practices researched by CAEL. 



What Early Results Show 

The Adult Learner Inventory (ALI) measures how important it is that an institution meets an adult 
student’s expectation and how satisfied the adult student is with the institution’s ability to meet the 
expectation. The difference between the importance of a given institutional practice or service and 
students’ satisfaction is the ALI “gap” score. A high “gap” score indicates challenges that an institution 
may want to address, while a low “gap” score indicates institutional strengths. 

In CAEL’s study, the correlation between re-enrollment percentages reported by the pilot 
institutions and the ALI “gap” scores is stunning. In short, what the data show generally is that 
colleges and universities with high “gap” scores have lower rates of re-enrollment, and conversely, 
institutions with low “gap” scores have higher re-enrollment rates. In order to demonstrate this 
correlation, Tables 2 and 3 list the ALI “gap” scores (converted onto a scale of 1-100) paired with 
the re-enrollment percentages (right-most column), from the top five and bottom five schools 
(based on re-enrollment rate). 
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Table 2. Adult Learner Inventory “Gap” Scores and Percentage of Re-enrolled Students for the Top 5 
Pilot Institutions (by Re-enrolled Percentage), with Mean Values (shown in bold in the bottom row). 



Outreach 


Life & 
Career 


Financing 


Assessing 

Outcomes 


Teaching/ 

Learning 


Student 

Supports 


Technology 


% 


7.00 


6.30 


11.20 


7.20 


4.90 


8.70 


6.30 


.99 


10.80 


12.40 


8.70 


10.50 


7.70 


13.00 


12.00 


.93 


4.60 


7.30 


5.50 


7.30 


4.70 


5.70 


5.30 


.92 


9.00 


12.50 


9.40 


10.80 


8.60 


12.00 


9.20 


.88 


8.70 


10.40 


11.20 


10.40 


6.70 


10.70 


10.30 


.87 


Mean = 8.02 


9.78 


9.20 


9.24 


6.52 


10.02 


8.62 


.92 



Table 3. Adult Learner Inventory “Gap” Scores and Percentage of Re-enrolled Students for the 
Bottom 5 Pilot Institutions (by Re-enrolled Percentage), with Mean Values (shown in bold in the 
bottom row). 



Outreach 


Life & 
Career 


Financing 


Assessing 

Outcomes 


Teaching/ 

Learning 


Student 

Supports 


Technology 


% 


20.00 


19.50 


17.30 


18.80 


16.50 


23.00 


21.80 


.61 


19.20 


22.00 


14.00 


20.30 


15.00 


20.30 


13.50 


.59 


9.50 


11.50 


11.20 


11.40 


8.30 


12.70 


8.20 


.50 


5.70 


9.10 


9.70 


8.20 


5.40 


8.70 


5.30 


.48 


17.10 


15.40 


19.90 


16.70 


12.40 


21.80 


17.90 


.48 


Mean = 14.30 


15.50 


14.42 


15.08 


11.52 


17.30 


13.34 


.53 



These initial results are encouraging; they are powerful initial evidence of the concurrent validity of 
the ALFI Assessment Tools with respect to student retention in the form of subsequent re-enrollment 
for more academic work at the same institution. In that respect, they bolster this step in the evaluation 
model — the impact of ALFI Assessment Tools on adult learners through positive student-level 
outcomes. Yet a closer examination of adult student retention is in order. 
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A Continuum of 
Adult Retention/Success 



In our evaluation of the pilot institutions’ scores from ALFI Assessment Tools, CAEL’s experience is 
that institutions that have high institutional scores on the Adult Learner Inventory (ALI) and the 
Institutional Self-Assessment Survey (ISAS) will concurrently have strong adult student retention and 
success rates because those institutions are proactively trying to remove barriers facing adult learners. 

The first challenge here is how to define retention of adult students in an appropriate and meaningful 
way. The key federal benchmark for college student retention is calculated on first-time-full-time 
student cohorts. Unfortunately, this standard is almost completely inapplicable for the vast majority of 
adult college students. Another part of the challenge is that traditional approaches measure retention 
in terms of traditional academic periods — in quarters or semesters. However, as we identify in the 
Principles of Effectiveness for Serving Adult Learners, the best adult learning programs acknowledge 
the special needs of adult learners by offering unique structures — such as accelerated learning formats, 
rolling start dates, competency-based curricula, and contract learning — to help adults balance their 
academic goals with their other responsibilities. For example, some institutions permit students to 
enroll in modularized courses as frequently as every two weeks. These special enrollment arrangements 
and schedules are designed to improve adult student access and retention, yet traditional retention 
measures based on quarters and semesters fail to capture the suitability and satisfaction experienced by 
adults engaged in quickly recurring, intensive, nontraditional program formats. For that reason, in our 
initial studies involving the pilot institutions, we correlated scores from the ALFI Assessment Tools 
with rates of re-enrollment rather than term-to-term persistence. 

Consideration of these issues lead to the conclusion that adult student retention should not be 
gauged upon continuous term-to-term enrollment, as is done for traditional students. Rather, an 
approach is needed that recognizes nontraditional formats and schedules. Adult student retention is 
fundamentally different than retention of traditional- age, residential students. Traditional students’ 
principal identity is that of student, even if they work (known as “students who work”), so even a 
temporary interruption of their studies disrupts that identity. By contrast, adult students’ principal 
identities are typically grounded in family and / or career, so the student role is complementary 
(otherwise known as “employees who study”); therefore, an interruption of studies may signify an 
adaptive not a disruptive response to life circumstances. 



Adult students generally show poor retention as measured traditionally because they often enroll in 
one course at an institution and do not immediately re-enroll at the next opportunity. There may 
be “adaptive” reasons for this: students may not re-enroll because they needed the course only to 
prepare for or to complete a requirement for a degree program at another institution or because 
they enrolled in order to gain specific knowledge or expertise for their career. From a traditional 
retention perspective, these students’ enrollment behavior might simply be seen as a “drop out” or 
“stop out” — and might be counted as such in traditional measures, even though, from their 
perspective, they were successful in reaching their academic goals. 

Measuring Adult Student Success 

Student success is a concept for which there has been far less consensus than retention. Many 
believe that degree completion represents the full measure of success. Seen from that perspective, 
retention is then an intermediate or temporary indicator of progress within an intact program of 
study — in other words, waypoints on the road to success. However, it is also believed that an adult 
student is successful when he or she reaches an individual goal, even if that is just the completion of 
a course or certificate, well short of a full degree. For example, an adult student may need just one 
course to transfer to another institution that offers programs that better fits his or her needs. Or to 
get a job promotion an adult student may need a few courses or a certificate, but not a degree. By 
having transfer credit accepted or by achieving new employment or higher earnings, the learner may 
also experience success even without formal program completion. 

Past and common measures of student success have often been based on measures that are more 
meaningful for traditional students than for adult learners. Good examples of such measures are 
grade point averages and time-to-degree. Because of the “stop out” phenomenon mentioned earlier, 
these measures can be void of meaning for adult students. An example of how such measures are 
inadequate for adults emerged at a large public university that participated in the piloting of the 
ALFI Assessment Tools. CAEL learned that most of the school’s adult students brought their 
required sixty credit hours of previous study from other institutions, but nearly a third of those 
students had originally attended this same university as “traditional” students, thus bringing forward 
credits that they had earned years earlier. Yet those students who had attended the university as 
traditional students confronted challenges when the grades that they earned during a previous phase 
in their lives were eventually merged into their current GPA calculation. Many ended up on 
academic probation because their grades during that previous experience at the university were not 
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as good as those that they currently earned as adults. Traditional approaches — such as assessing 
academic performance from the distant past as part of current academic progress — may seem on 
the surface to be objective, but they are not true indicators of the student’s potential success as an 
adult learner. In this example, the most recent work is the true indicator of how the student should 
be regarded and not the cumulative record. 

A critical element within measures of retention as well as many measures of success is that of time 
frame. Fewer adults proceed into and through their studies with the same course-load level or 
semester-to-semester continuity compared to traditional students (eighteen- to twenty-four-year-olds). 
Because of competing obligations, adults generally take fewer classes, with more interruptions, over 
longer periods of time compared to younger students. There is no “right” or universal standard for 
intervals of time to use when measuring retention or of success when viewed as progression towards 
graduation. Too often, to acknowledge a “retention problem” for adult learners is to make unrealistic, 
if not unfair comparisons, to younger students for whom our expectations are different. 



Retention and Success 
New Yardsticks 



CAEL convened a team of subject-matter experts to discuss alternative definitions of retention and 
success for adult students. CAEL then used these definitions to obtain data from as many of the 
original pilot institutions as possible and to see how well these measures correlated with their scores 
from the ALFI Assessment Tools. CAEL expects that these alternative measures of retention and 
success may be suitable for exploration in other contexts that involve adult learners, but they are 
not presumed to be the best or only approaches for use by other researchers. 

The new conceptual framework that CAEL adopted for purposes of our research reported here 
begins with these high-level definitions: 

■ Retention identifies the frequency with which adult students, having enrolled in 
credit-bearing work, re-enroll for more credit-bearing work within one year. 

■ Success is identified by the frequencies with which adult students do the following, 
according to predetermined thresholds and time intervals: 

□ complete one or more credit-bearing courses, 

□ accumulate specified levels of credits applicable toward certificates or degrees, 

□ earn academic credentials or transfer credits to other institutions, 

□ avoid academic probation or dismissal. 
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Thus, the success measure presumes retention but focuses upon academic progress that includes 
a dimension of satisfactory academic performance. 

Because CAEL wanted comparable data across all pilot institutions that would correlate with the 
institutional and student survey results obtained during 2002-2003, CAEL provided the pilot 
schools with the following specific time frame and enrollment activity periods: 

“Timeframe - Activity (defined below) on adults students who enrolled in certificate or undergraduate 
degree programs during school year 1998-1999 (from July 1, 1998 but no later than June 30, 1999) 
which occurred anytime through the end of school year 2003-2004 (by June 30, 2004). This number 
should include all first-time adult student enrollees, including adults who have academic credit from 
other institutions but are enrolling at your institution for the first time. This number is the ‘base’ 
number for calculating measures of activity anytime over the specified five-year period; the activities 
measured are given below. Please note: Do not include graduate students.” 

In conjunction with this specified time frame, pilots were then requested to provide data as it relates 
to the following academic progression or “milestones:” 

“Of the adult students that matriculated at your institution anytime during the 1998-1999 aca- 
demic year (as defined above by timeframe): 

a) How many of these students completed at least one Title IV eligible course or course 
equivalent, by the end of the 2003-2004 school year? A course equivalent may be credits 
earned through CLEP, challenge exams, or other prior learning assessments. Count each 
student only once. If eligible to be counted in this category, the same students may also be 
counted again below. 

b) How many of these students completed at least twelve academic credit hours or earned 

a credential such as a certificate by the end of the 2003-2004 academic year? Count each 
student only once. If eligible to be counted in this category, the same students may also be 
counted again below. 

c) How many of these students completed thirty academic credit hours or completed 
a degree by the end of the 2003-2004 academic year? Count each student only once. 

If eligible to be counted in this category, the same students may also be counted again below. 

d) How many of these students continued their academic work at another postsecondary 
institution? This transfer may have occurred in one of two ways: 1) a lateral move to another 
academic program or 2) a move to an institution that is a different type than your own (Ex: 
two-year to four-year). Evidence of transfer may be an academic transcript request or other 
systematic tracking means available to your institution. Count each student only once.” 
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A second set of indicators relates to satisfactory academic performance: 

“Of the adult students that matriculated at your institution any time during the 1998-1999 
academic year (the same group as above): 

a) How many of these students were placed on academic probation (as defined by your 
institution) at any time through school year 2003-2004? 

b) How many of these students were subject to academic dismissal at any time through school 
year 2003-2004? 

■ Note: Since students who have been dismissed usually were first placed on academic 
probation, students counted in ‘option b’ will duplicate many cases of students counted 
in ‘option a.’ 

■ Note: These data should be available as part of your institution’s policies to comply with 
Title IV eligibility requirements.” 

While imperfect in some respects, the virtue of these definitions as advanced by the subject-matter 
expert team that CAEL assembled is that nearly every institution of higher education is capable 
of or actually reported student work in credit hours, and nearly all will also have provisions for 
measuring satisfactory academic progress, upon which continuing student eligibility for federal 
student aid programs depends. 



Results and Analysis 

This section will report in particular on academic progression (credit hour production) and 
satisfactory academic performance (probation/dismissal) among the pilot institutions. Eighteen 
pilot institutions provided credit hour data and seventeen institutions provided probation/dismissal 
data to CAEL. Since this study was developed after the pilot project, there was no previous 
arrangement or obligation for institutions to participate. Since a relatively small number of 
participating institutions provided data, caution must be exercised in placing too much reliance or 
value upon findings from this small sampling. 

The credit hour and probation/dismissal indicators showed the expected intercorrelations, which 
lends confidence into their use as outcome scales. Course completions correlated with completion 
of twelve credit hours as r=.698 (p=.001) and with completion of thirty credit hours as r=.557 
(p=.0l6), while the completion of twelve credit hours with thirty credit hours was r=.943 (p<.001). 
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Similarly for the measures of satisfactory academic performance, probation correlated with dismissal 
as r=.709 (p<.001). There were no significant correlations between measures of credit hour 
production and probation/dismissal. 

Only a few of the student outcome measures showed statistically significant relationships to the 
scores from the ALFI Assessment Tools in this small sampling of CAEL’s pilot institutions. Three of 
the scores reported on data from the Institutional Self-Assessment Survey (ISAS) showed negative 
relationships to the academic probation measure, such that higher scores of these variables accompany 
lower incidence rates of students on probation. Those scales are the Assessment of Learning Outcomes 
(r= -.762, p<.001), the Teaching/Learning Process (r= -.771, p<.001), and Student Support Systems 
(r= -.537, p=.026). While CAEL would have been pleased to observe other significant negative 
relationships between scores from the ALFI Assessment Tools and probation/dismissal, it must also 
be recognized that these three subscales are the only ones directly relevant to the academic process. 
There is no theoretical or practical reason to believe that institutional activities related to the other 
subscales (such as Outreach, Finance, or Strategic Partnerships) should be expected to have any 
material relationship to student academic performance. 

With respect to academic probation, a statistically significant correlation emerged between the 
ISAS scale for the Teaching/Learning Process and dismissal — again negative — such that higher 
Teaching/Learning Process scores are inversely related to occasions of academic dismissal (r= -.505, 
p=.039). As before, no other significant correlations to other scores from the ALFI Assessment 
Tools were observed. 

Regarding the Adult Learner Inventory (ALI), the Teaching/Learning Process scale also showed some 
relationship to student credit hour production. The Importance score (but not the Satisfaction score) 
for Teaching/Learning Process is positively correlated with the scales measuring student credit hour 
production at the twelve-hour level (r=.527, p=.025) and at the thirty-hour level (r=.504, p=.033). 
Here again, this positive relationship is “intuitively correct” insofar as effective instructional methods 
for adults should be expected to coincide with a higher incidence of earned credit hours among the 
adults. No other statistically significant relationships emerged with other ALI scores and the academic 
progression scales used for this study. 

It should also be noted that only a few institutions that participated in this study could provide 
data on the number of students who continued their academic work at another institution. It is 
striking that at a time when it is so common for students to transfer that institutions do not follow 
this trend. CAEL believes this indicates an additional need in the higher education community to 
consider data collection and studies. 
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Discussion and Implications 

A full-scale rigorous test of the new, alternative definitions of retention and success measures that 
are appropriate for adults as correlated with the institutional effectiveness measures that the scores 
from the ALFI Assessment Tools capture will require tracking a large cohort of institutional users 
of the ALFI Assessment Tools over a number of years with their corresponding student outcomes. 

Such a test will also require suitable control measures to monitor external or environmental influences 
that might otherwise explain institutional and student behavior. Needless to say, a full-scale test of the 
predictive validity of the ALFI Assessment Tools will be an intensive undertaking. Nonetheless, the 
work of the past few years in developing the Tools provides immediate and strong evidence 
underpinning the justification of the model. 

The newly proposed definitions used for this research merit further testing and refinement. They 
likely represent the best available compromise between commonly available, reliable measures across 
a wide variety of institutions and the highly particular and immediate outcomes that characterize 
the intentions of adults students (i.e., Does this learning serve my individual goals? Can I use this 
learning tomorrow if not sooner?). For its part, CAEL will continue to be interested in continuing 
to participate in the development and refinement of better measures of the effectiveness of higher 
education not only at the student level but at the institutional level as well. 



13 




REFERENCES 



Flint, Thomas A. & Associates. (1999). Best Practices in Adult Learning: A CAEL/APQC 
Benchmarking Study. New York: Forbes Custom Publishing. 

Flint, T A., Zakos, P., and Frey, R. (2002). Best Practices in Adult Learning: A Self-Evaluation 
Workbook for Colleges and Universities. Dubuque: Kendall/Hunt Publishing Company. Order 
through Kendall/Hunt Publishing: 1-800-228-0810. 

National Center for Education Statistics. (2002). The Condition of Education 2002. 

(NCES 25-39). Washington, DC: U.S. Department of Education. 



14 




4>cael 



55 East Monroe, Suite 1930 • Chicago, IL 60603 • 312.499.2600 • FAX 312.499.2601 • www.cael.org 



